Projector type display apparatus

ABSTRACT

There is provided a safe image display apparatus for projecting an image on a screen that can minimize a stimulus to eyes, retinae and optic nerves from projected light entering into the eyes directly. By comparing a displayed image captured by a camera with an input image signal, a changed area is detected as a difference area and, then, a projected image signal is generated wherein a picture area image signal in the input image signal that corresponds to this difference area is masked by a black signal. The obtained projected image signal is projected onto the screen.

RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.10/619,168, filed on Jul. 15, 2003 now U.S. Pat. No. 6,945,653.

BACKGROUND OF THE INVENTION

The present invention relates to a safety improvement technique in aprojector type image display apparatus that is configured to displayvisual information by projecting it on a screen.

In a conventional liquid crystal projector that is configured to projectan image on a screen in front of the projector, as there is a risk inthat eyes of a speaker or other persons on the platform are stimulatedby strong light when the persons enter into the path of the lightprojected from the projector to the screen and look toward theprojector, such risk must be prevented by indicating precautions on themain body of the projector and so on.

Further, as described in Japanese Patent Laid-open No. 2000-305481 andJapanese Patent No. 2994469, there has been proposed a method forpreventing such risk by providing detection means for detecting whetherthere is an object in a projection path and control means forcontrolling an amount of light from a projection lens, wherein lightintensity from the projection lens is reduced when the object isdetected.

SUMMARY OF THE INVENTION

However, in the conventional displaying method for alarming the risk, itmay still be likely that any person enters into the projection path andsuch risk can be accommodated merely by calling for further attention.

On the other hand, in the patent document 1, there is a problem in thatthe light intensity of the image in the vertical direction including theperson is reduced because the area of the person entering into theprojection path is detected and, in the patent document 2, in turn,there is a problem in that, when the person area is extracted and thenmasked by using various sensors such as, in particular, a CCD sensor,person extraction accuracy may be degraded and, therefore, falsedetection may occur due to insufficient exposure of the CCD.

In view of the above problems, it is an object of the present inventionto provide an image display apparatus that can reduce glare by reducingprojected light entering into eyes while keeping a picture projected ona screen during a presentation using the projected image by a speaker onthe platform even when (1) someone enters into the projection pathaccidentally, (2) the speaker standing in the projection path turnshis/her face to the projector, or (3) a picture is displayed in anenlarged manner or trapezoidal distortion is compensated because ofsetting of the projector and the screen.

In order to solve the above problems, according to the presentinvention, there is provided a projector type display apparatus forapplying light from a light source to a picture display device (a lightvalve) and projecting outgoing light from said picture display deviceonto a screen in an enlarged manner having: display device control meansfor controlling said picture display device based on an input imagesignal; a camera for capturing an image projected on said screen;detection means for detecting an area projected on said screen; imagesize control means for changing the captured image from said camera intoa predetermined image size; and means for extracting a person area insaid captured image the size of which is changed by said image sizecontrol means; wherein said display device control means controls sothat an area on said image display device corresponding to said personarea is substituted by predetermined data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an exemplary system configuration to whichthe present invention is applied;

FIG. 2 is a diagram showing a first embodiment of an image displayapparatus according to the present invention;

FIG. 3 is a diagram showing a second embodiment of an image displayapparatus according to the present invention;

FIG. 4 is a diagram showing a third embodiment of an image displayapparatus according to the present invention;

FIG. 5 is a diagram showing a fourth embodiment of an image displayapparatus according to the present invention;

FIGS. 6A–6G are diagrams for schematically showing image information ineach image processing section in the embodiments of the presentinvention; and

FIGS. 7A–7C are diagrams for schematically showing image information ineach image processing section in the embodiments of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the drawings.

FIG. 1 is a schematic diagram showing an exemplary system configurationin which a front projection type projector apparatus according to thepresent invention, a screen and a speaker on the platform are disposed.Further, FIG. 2 is a block diagram showing a first embodiment of animage processing unit according to the present invention. Still further,FIGS. 6A–6G are schematic diagrams showing image information in eachprocessing block.

Here, an embodiment using a light valve type display device thatcontrols an amount of light transmitted by a liquid crystal or reflectedby a reflecting mirror according to a gray-scale level of each pixel inimage information will be described. The display device is not limitedto a particular type and it may be any display device such as athree-pipe type or a CRT type. Further, though the case in which oneimage display device as shown in FIG. 2 is included is described, thepresent invention may also be applied to the case in which the imagedisplay device is comprised of separate display panels for each of theRGB signals similarly.

Further, though the display panel of XGA (1024×768×RGB) resolution andthe camera of SXGA (1280×1024×RGB) resolution are assumed in thefollowing description for the sake of simplicity, the display paneland/or the camera of other resolution or a plurality of the displaypanels and/or the cameras may be used similarly, of course. The cameraof either a CCD type or a CMOS type may be used similarly. Here, it isassumed that an image frame frequency Fv is, for example, 60 Hz(T1=about 16.67 msec., frame numbers f0, f1, f2, f3, . . . ) in thefollowing description.

In FIG. 1, there are shown a front projection type projector 1, aprojection lens 2, a projection: path 3, a screen 4, a camera 5, acapture area 6 by the camera 5, a taking path 7, a person (a speaker onthe platform) 8, and a shade 9 of the speaker on the platform. Further,FIG. 2 shows a first embodiment of an image display unit in theprojector 1, wherein there are shown a video signal input 10, a videoprocessor 11, a camera interface 12, a projection area detector 13, ascaling unit (image size change unit) 14, an image comparator 15, adifference area generator 16, a mask image generator 17, an image signalcombining unit 18, a lamp (light source) 19, and a light valve (imagedisplay device) 20. FIGS. 6A–6G are diagrams for schematically showingimage information in each image processing section.

Light is projected from the projection lens 2 through the projectionpath 3 onto the screen 4 to reproduce the projected image. The camera 5obtains captured image information taken in a capture area enclosing theprojected light area through the taking path 7. Here, when theprojection lens 2 and the camera 5 are disposed in positions close toeach other, the projection path and the taking path substantially agreewith each other and, therefore, a distortion of the captured imagesubstantially agrees with a distortion of the image seen through theprojection lens. Of course, these distortions may not always agree witheach other when the projection lens 2 is apart from the camera 5, lensesof different properties are used, or the projected position is changedby adjusting the position where the projection lens 2 is disposed. Forexample, the captured image may be as shown in FIG. 7A. Even in suchcase, according to the present invention, the person area can beextracted properly by an image scaling process in the scaling unit 14 asdescribed below.

When the person 8 enters into the projection path 3, the person 8intercepts the projected image and appears as the shade 9 on the screen.On the other hand, the camera 5 captures the projected image includingthe person 8 entering into the taking path 7. Here, it is to be notedthat the shade 9 is not captured as discussed above.

Next, a processing operation in the projector 1 will be described indetail. First, the video processor 11 generates displayed imageinformation (FIG. 6B) obtained by processing input image informationfrom the video signal input 10 so that the input image informationconforms to the display format of the light valve 20. For example, theinput image as shown in FIG. 6B is processed to be in XGA size andpixels located at each corner of the image are designated as I, J, K andL, respectively.

Further, though R, G and B are processed identically in this embodiment,R, G and B may be processed separately or complementary color signalsmay be used, of course, without departing from the present invention.

When each of R, G and B light is incident on the light valve 20, thelight having the intensity according to a gray-scale level of theprojected image information determined by FIG. 6B per every pixel istransmitted or reflected and, then, the outgoing light is projected onthe screen 4 through the projection lens in an enlarged manner. This isa typical configuration used in liquid crystal projectors and,therefore, its detailed description is omitted.

On the other hand, the camera 5 obtains the captured image as shown inFIG. 6A. Here, pixels located at each corner of the captured image ofSXGA size that is the camera's resolution are designated as E, F, G andH, respectively, and pixels located at each corner of the area where theprojected image are indicated inside the captured image are designatedas A, B, C and D, respectively.

The camera interface 12 corrects the captured image for properties ofeach camera, adjusts the color balance, the hue and the like of thecaptured image and outputs the captured image information (FIG. 6A). Theprojection area detector 13 considers the captured image information andgenerates area information by determining the area in which theprojected image is included. Here, the determination method describedhere is not intended to limit the present invention in any way and thedetermination may be performed by comparing the captured image with anadjacent image or former image information. Further, the determinationmay be performed according to the properties of the projection lens 2and the camera 5 or the area of the captured image may be defined inadvance according to the magnification of the projection lens 2 and thelike. Still further, the area information may be fixed by changing themagnification of the capture lens of the camera 5 according to themagnification of the projection lens 2 or, alternatively, the areainformation may be specified from outside.

The scaling unit 14 performs a scaling (enlargement or reduction)process of the captured image information according to the areainformation so that the positions of the pixels A, B, C and D agree withthe positions of the pixels A′, B′, C′ and D′ and outputs it asdisplayed area extraction image (FIG. 6D). More specifically, thecaptured image as shown in FIG. 6A is turned into the scaling image asshown in FIG. 6C. Then, as shown in FIG. 6D, only the image of theprojected area is extracted from the scaling image as the displayed areaextraction image.

Here, an area disposed between the pixel positions E′, F′, G′ and H′corresponding to SXGA size and the pixel positions A′, B′, C′ and D′ isdefined as the out of projected area image. Here, horizontal or verticaltrapezoidal correction may be performed as the scaling process so as toimprove flexibility in disposing the projection lens 2 and the camera 5.

The image comparator 15 generates disagreement information by comparingthe image information of the displayed area extraction image with theinformation of the projected combined image per every pixel. In thiscase, the disagreement information may be generated by featurecomparison in information average values of surrounding pixels, featurecomparison in information variation between pixels, or from informationabout the person's motion by comparison between frames. Thisdisagreement information is, for example, defined as 1 in the area wherethe agreement is found and 0 in the area where the disagreement isfound.

On the other hand, an amount of compensation and adjustment of thecamera interface 12 may be determined according to the comparison resultso as to improve comparison accuracy. Further, the pixels to be comparedmay be thinned out. Still further, in addition to the person area,facial information may also be distinguished. Here, the areadetermination method described above is not intended to limit thepresent invention in any way and any known method may be used.

Next, the difference area generator 16 generates differential areainformation when it is determined that there is the person in the areaon the screen including the disagreeing pixels based on the disagreementinformation. Here, the entire person area, the face area, the eye area,an area of a particular shape and the like may be selected as thedifferential area. Further, former differential area information may beheld every frame period Fv and, then, new differential area informationmay be generated by using the former differential area information and,further, the shape of the differential area may be fixed according tothe former information and, then, a new differential area may begenerated by area movement.

The mask image generator 17 generates masked image information (FIG. 6E)by reducing the gray-scale level of the pixels indicated by thedifferential area information in the displayed image information orreplacing such pixels by either a particular color having a lowgray-scale level or black. The masked image information may be any imagethat is less stimulating visually such as an image in which the pixelsin the disagreement area is replaced by black signals or gray-scalesignals of a predetermined level as shown in FIG. 6E and the maskedimage information is not limited to any particular form.

The image signal combining unit 18 superimposes the masked imageinformation on the displayed image information to generate projectionimage (FIG. 6F).

Further, the image comparator 15 detects an area in which changesbetween frames can be found by information comparison or a motion areaby using the image information out of the projected area in the capturedimage information shown in FIG. 6G. Here, such area may be determined asthe person area and the shape of the differential area may be correctedby using this person area. Further, the entry into the projection path 3may be detected in advance by comparison with the former frame.

Here, even when the images are distorted unevenly because the projectionlens 2 is apart from the camera 5 or each of these has different lensproperties, for example, in the case of the captured image shown in FIG.7A, it goes without saying that the process described above may bepossible by performing the trapezoidal correction in the scaling unit 14to generate the image shown in FIG. 7C.

According to the first embodiment described above, when any objectenters into the projection path, it is possible to reduce the intensityof the light projected on the object only. Further, when the personentering into the projection path is the speaker on the platform, as theimage is projected on the screen normally, the action of the speakerusing the screen may not be interrupted and the speaker can watch thestate of an audience sitting at the side of the projector easily. It canimplement a presentation environment that is comfortable and impressive.

Further, according to the first embodiment, even when the image isdisplayed in an enlarged manner by the projector or the trapezoidaldistortion in the displayed image is corrected, it is also possible toshade the area projected onto the speaker on the platform.

Still further, according to the first embodiment, by displaying abackground image by the projector, a new application such as to improvea visual effect centering around a person can be implemented.

Next, a second embodiment of the present invention will be describedwith reference to FIG. 3. Here, elements having reference numeralsidentical to those described in the first embodiment have substantiallysimilar features, description of which will be, therefore, omitted.

FIG. 3 shows a configuration in which a capture timing controller 21 isadded to the configuration in FIG. 2.

According to this configuration, the capture timing controller 21 countsa frame synchronization signal Fv and permits to capture an image onlyduring one frame period in N frame periods, wherein the number of N iseither fixed or settable, for example, N=60. The camera interface 12captures the image from the camera 5 during the frame period when thecapture is permitted by a capture permission signal (for example, it islabeled as 1 when the capture is permitted and 0 when the capture isheld) and the capture is stopped during the frame period when thecapture is prohibited.

In this case, a former captured image may be retained in the frameperiod when the capture is prohibited. Further, the frame period whenthe capture is permitted may be further time-divided.

Here, the value of N such as N=60, 125, 250 and so on may be selectedbased on a camera shutter speed (for example, 1/60, 1/125, 1/250 and soon) and, further, the frame period when the capture is permitted may betime-divided into 125/60 (250/60) so that it becomes equivalent to N=125(250).

On the other hand, a similar control signal (for example, it is labeledas 1 when the image is unchanged and 0 when the image is masked) is alsosupplied to the image signal combining unit 18 so that the displayedimage is selected to be projected during the frame period when thecapture is permitted (when the image is unchanged). Further, during theframe period when the capture is prohibited (when the image is masked),an image on which the masked image is superimposed is selected. In thiscase, the displayed image may be projected in a shorter period that isdefined by further time-dividing the frame period when the capture ispermitted.

At this time, the image comparator 15 is configured to compare thedisplayed area extraction image from the scaling unit 14 with thedisplayed image from the video processor 11.

Further, lighting time of a flash lamp used for the capture by thecamera may be set to a value on the order of one several thousandthsseconds.

Here, it is to be noted that a laser radiation safety standard isdefined by the IEC 60825-1 standard or, in Japan, by the JIS C6802 (aradiation safety standard for laser products). For example, a laserpointer having an output not more than 1 mW He—Ne (Helium-Neon)regardless of a laser diameter or a single pulse (a pulse width of 10⁻⁷s.) visible laser having an output not more than 3 W is defined as areference level at which retinae are not damaged even when the eyes areexposed to the laser light during a nictitation or for 0.25 s. or safetyis assured even when any supplementary optical system such as binocularsis used at the worst.

Though the frame period N and the amount of emitted light are notspecified clearly in the above description of this embodiment, thesevalues may be determined according to how the person standing in theprojection path suffers glare and, at the same time, according to thestandards described above.

According to the second embodiment described above, even when the personis moving, the accuracy of the masked area can be ensured periodicallywhile the person in the projection path may not suffer the glare.

Further, though the case in which the image signal combining unit 18selects the displayed image during the frame period when the capture ispermitted (when the displayed image is not changed) or the image onwhich the masked image is superimposed during the frame period when thecapture is prohibited (when the displayed image is masked) as the imageto be projected has been described in the second embodiment, an image inwhich the intensity level of an area corresponding to the person area isreduced may be selected during the frame period when the capture ispermitted. In this case, in a manner similar to the first embodiment,the image comparator 15 is configured to compare the displayed areaextraction image from the scaling unit 14 with the image having thereduced intensity level from the image signal combining unit 18.

Next, a third embodiment of the present invention will be described withreference to FIG. 4. Here, elements having reference numerals identicalto those described in the first embodiment have substantially similarfeatures, description of which will be, therefore, omitted.

In FIG. 4, there are shown an infrared camera 23 and a person areadetector 22. Here, infrared light is split from the light source 19 andprojected through the projection lens 2 or infrared light is generatedfrom a separate light source that is not shown and projected onto thescreen. Further, the infrared light projected onto the screen mayconform to the image display area substantially.

According to this configuration, the infrared camera 23 catches thelight reflected from the person standing in the projection path 3 withthe infrared light reflected from the screen 4 and conveys theinformation to the camera interface 12. The person area detector 22extracts the background area and the person area from the captured imagebased on, for example, the difference in an amount of reflection. Atthis time, the information about the magnification of the projectionlens 2 is supplied to the projection area detector 13 so as to determinethe projected area on the screen.

According to the third embodiment described above, it is possible toextract the person with high accuracy and without dependence on thedisplayed image information and, further, cost reduction can beimplemented easily because the infrared light can be obtained bysplitting it from the white light source.

Next, a fourth embodiment of the present invention will be describedwith reference to FIG. 5. Here, elements having reference numeralsidentical to those described in the first and third embodiments havesubstantially similar features, description of which will be, therefore,omitted.

In FIG. 5, there is shown a thermo-camera 24. Here, the light source issimilar to the one in FIG. 2.

According to this configuration, the thermo-camera 24 senses thedistribution of the temperature emitted from the person standing in theprojection path and the distribution of the temperature emitted from thescreen 4 and conveys the information to the camera interface 12. Theperson area detector 22 extracts the background area and the person areafrom the captured image based on the difference in the temperaturedistribution. At this time, the information about the magnification ofthe projection lens 2 is supplied to the projection area detector 13 soas to determine the projected area on the screen.

According to the fourth embodiment described above, the image displayapparatus with high accuracy can be implemented without dependence onthe displayed image information.

As described above, according to the present invention, even when theperson turns his/her face in the direction of the light projected fromthe projector, an amount of the light entering into the eyes can bereduced.

1. A projector type display apparatus for applying light from a lightsource to a picture display device and projecting outgoing light fromsaid picture display device onto a screen in an enlarged mannercomprising: a video processor which generates a display pictureinformation; a camera interface which takes in a captured image capturedby a camera and outputs as captured image information; an image sizecontroller which outputs an area determined as a projected image as adisplayed area extraction image; an image comparator which generatesdisagreement information at least in a part of a person area bycomparing the image information of said displayed area extraction imagewith the information of projected combined image; a difference areagenerator which generates differential area information based on saiddisagreement information; a mask image generator which generates maskedimage information based on said differential area information; and animage signal combining unit which superposes said masked imageinformation on said displayed image information and outputs thedisplayed image information as a newly projected combined imageinformation.
 2. The projector type display apparatus according to claim1, wherein said image size controller controls and changes said capturedimage information into a predetermined image size.
 3. The projector typedisplay apparatus according to claim 1, wherein said image sizecontroller performs trapezoidal correction.
 4. The projector typedisplay apparatus according to claim 1, wherein said image comparatorgenerates said disagreement information in the entire area of saidperson area.
 5. The projector type display apparatus according to claim1, wherein said image comparator generates said disagreement informationin a face area of said person area.
 6. The projector type displayapparatus according to claim 1, wherein said image comparator generatessaid disagreement information in an eye area of said person area.
 7. Theprojector type display apparatus according to claim 1, wherein saidimage comparator generates the disagreement information in a particularshape area of said person area.
 8. A projector type display apparatusfor applying light from a light source to a picture display device andprojecting outgoing light from said picture display device onto a screenin an enlarged manner comprising: a video processor which generates adisplay picture information; a camera which captures an image projectedonto the screen; a camera interface which takes in a captured imagecaptured by a camera and outputs as captured image information; an imagesize controller which outputs an area determined as a projected image asa displayed area extraction image; an image comparator which generates adisagreement information at least in a part of a person area bycomparing an image information of said displayed area extraction imagewith an image information of projected combined image; a difference areagenerator which generates differential area information based on saiddisagreement information; a mask image generator which generates amasked image information based on said differential area information;and an image signal combining unit which superposes said masked imageinformation on said display picture information and outputs said displaypicture information as a newly projected combined image information. 9.The projector type display apparatus according to claim 8, furthercomprises a capture timing controller which controls a capture timing ofsaid camera and controls said picture display device based on adisplayed video signal input at least during a period when said cameracaptures a image projected onto said screen.
 10. The projector typedisplay apparatus according to claim 8, wherein said image comparatorgenerates said disagreement information in the entire area of saidperson area.
 11. The projector type display apparatus according to claim8, wherein said image comparator generates said disagreement informationin a face area of said person area.
 12. The projector type displayapparatus according to claim 8, wherein said image comparator generatessaid disagreement information in an eye area of said person area. 13.The projector type display apparatus according to claim 8, wherein saidimage comparator generates said disagreement information in a particularshape area of said person area.
 14. A projector type display apparatusfor applying light from a light source to a picture display device andprojecting outgoing light from said picture display device onto a screenin an enlarged manner comprising: a detector which detects an areaprojected on said screen; an image size controller which takes in acaptured image from a camera and outputs a displayed area extractionimage in a first predetermined image size, based on detection resultsfrom said detector; a video processor which generates a display pictureinformation from a video signal input so that said display pictureinformation conforms to display format of said picture display device;an image signal combining unit which superposes an area corresponding toat least a part of a person area included in said displayed areaextraction image, as being a masked image area, on said display pictureinformation.
 15. The projector type display apparatus according to claim14, wherein said image size controller changes said captured image intoa second predetermined image size and generates said displayed areaextraction image from said captured image of said second predeterminedimage size.
 16. The projector type display apparatus according to claim14, wherein said image size controller performs trapezoidal correctionagainst said captured image and generates said displayed area extractionimage from said captured image being performed said trapezoidalcorrection.
 17. The projector type display apparatus according to claim14, wherein said image signal combining unit extracts the entire area ofsaid person area as said masked image area.
 18. The projector typedisplay apparatus according to claim 14, wherein said image signalcombining unit extracts a face area of said person area as said maskedimage area.
 19. The projector type display apparatus according to claim14, wherein said image signal combining unit extracts an eye area ofsaid person area as said masked image area.
 20. The projector typedisplay apparatus according to claim 14, wherein said image signalcombining unit extracts a particular shape area of said person area assaid masked image area.